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Twelve rats made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that 
was delivered immediately after a response and a larger reinforcer (B) that was delivered after a delay 
which increased or decreased by 20% depending on the subject’s choices in successive blocks of trials. 
In two phases of the experiment (100 sessions and 40 sessions), reinforcer sizes were selected which 
enabled theoretical parameters expressing the rate of delay discounting and sensitivity to reinforcer size 
to be estimated from the ratio of the indifference delays obtained in the two phases. Indifference delays, 
calculated from adjusting delays in the last 10 sessions of each phase, were shorter when the sizes of A 
and B were 14 and 25 |il of a 0.6 M sucrose solution than when they were 25 and 100 |il of the same 
solution. The ratio of the indifference delays was significantly smaller than that predicted on the basis of 
an assumed linear relation between reinforcer size and instantaneous reinforcer value, consistent with a 
previous proposal that this relation may be hyperbolic in form. Estimates of the rate of delay 
discounting based on the ratio of the two indifference delays (mean, 0.08 s -1 ) were similar to values 
obtained previously using different intertemporal choice protocols. Estimates of the size-sensitivity 
parameter (mean 113 |ll) were similar to estimates recently derived from performance on progressive- 
ratio schedules. In both phases of the experiment, adjusting delays in successive blocks of trials were 
analyzed using the Fourier transform. The power spectrum obtained from individual rats had a 
dominant frequency that corresponded to a period of oscillation of the adjusting delay between 30 and 
100 trial blocks (mean, 78). Power in the dominant frequency band was highest in the early sessions of 
the first phase and declined with extended training. It is suggested that this experimental protocol may 
have utility in neurobehavioral studies of intertemporal choice. 

Key words: intertemporal choice, adjusting-delay schedule, delay discounting, reinforcer magnitude, 
Fourier transform, lever press, rat 


In an intertemporal choice schedule, the 
subject chooses between reinforcers that differ 
with respect to their sizes and delays. For 
example, a subject may be confronted with two 
operanda, A and B; if it responds on A, a small 
reinforcer will be delivered after a short delay, 
whereas if it responds on B, a larger reinforcer 
will be delivered after a longer delay. 

According to one model of intertemporal 
choice behavior (Mazur, 1987, 2001, 2006), 
the reinforcing value of each outcome is a 
declining hyperbolic function of the delay 
interposed between the response and the 
primary reinforcer. 
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where q A and q B are the sizes of the two 
primary reinforcers, d A and d B are the delays 
associated with each alternative, and K is a 
parameter expressing the rate of delay dis¬ 
counting. It is assumed that when faced with a 
choice between A and B, the subject selects the 
outcome that has the higher overall value at 
the moment of choice. It should be noted that 
V refers not to the value of the primary 
reinforcer, but to the overall value of the 
conditioned reinforcing stimuli that are pres¬ 
ent at the moment of choice. Thus Equations 
la and lb do not imply that the primary 
reinforcer itself is devalued as a function of 
delay (Mazur, 1995). 

Equations la and lb imply direct propor¬ 
tionality between value and the size of a 
reinforcer. However, there is emerging evi¬ 
dence for a nonlinear relation between value 
and reinforcer size (Mazur & Biondi, 2009; 
Rickard, Body, Zhang, Bradshaw, & Szabadi, 
2009), in keeping with the economic concept 
of diminishing marginal utility (Killeen, 2009; 
Pine, et al., 2009). Ho, Mobini, Chiang, 
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Bradshaw and Szabadi (1999) proposed a 
modification of Equations la and lb to take 
into account the possibility of such a nonlinear 
relation. Ho et al. (1999) posited a hyperbolic 
relation, in which the effect of reinforcer 
magnitude on value is modulated by a single 
“size-sensitivity parameter”, Q: 

y A = 1 • 1 

1 + a/q A 1 + K.d A ' 

V B =^ _. 

1 A Q_/q B 1 + K.d B (2a, 2b) 


By experimentally manipulating the delays 
and/or sizes of A and B, a point of indiffer¬ 
ence may be determined at which the subject 
shows no preference for either outcome. It is 
generally assumed that indifference implies 
that Va = V B . Equating the right-hand sides of 
Equations 2a and 2b and rearranging the 
terms yields the following linear relation 
between the indifference delay to the larger 
reinforcer, d B <z,o), and the delay to the smaller 
reinforcer, dA (Ho et al., 1999): 


d B ( 50) 


1 

K 


Q/qA-Q/qB 


i + Q/qs _ 
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i + Q/ qA 
1 + Q/q B ' 


( 3 ) 


This relation can be used to examine the 
effects of neurobiological interventions on the 
hypothetical processes of delay discounting 
and sensitivity to reinforcer size. For example, 
if the sizes of Reinforcers A and B (q A and q B ) 
are held constant and indifference delays are 
determined for a series of delays to Reinforcer 
A (d A ), a change in the slope of Equation 3 
induced by a cerebral lesion implies a change in 
Q (i.e., a change in sensitivity to reinforcer size), 
whereas a change in the intercept without a 
concomitant change in slope implies a change 
in K (i.e., a change in the rate of delay 
discounting, Ho et al., 1999; Mazur, 2006). 
Using this approach it has been found that 
excitotoxic and dopamine-depleting lesions of 
the orbital prefrontal cortex increase both Q 
and K (Kheramin, et al., 2002, 2004), whereas 
destruction of the core of the nucleus accum- 
bens or disconnection of the nucleus accum- 
bens from the orbital prefrontal cortex produc¬ 
es a selective increase in K (Bezzina et al., 2007, 
2008). Equation 3 has also been applied 
successfully in experimental studies of inter¬ 


temporal choice in humans (Hinvest & Ander¬ 
son, 2010; Liang, Ho, Yang, & Tsai, 2010). 

A significant practical difficulty with the 
application of Equation 3 is the length of time 
needed to collect sufficient data to fit the 
linear functions, because each value of d B ( 50 ) is 
derived from steady-state performance using a 
different value of d A , which, in the case of 
animal subjects, may require 40-60 training 
sessions, and five or six d B ( 5 o)/ dA pairs are 
needed to obtain a reliable linear function 
(see, for example, Kheramin et al., 2002). 
Since neurobehavioral experiments involving 
cerebral lesions typically entail two or more 
groups of 10 or more subjects, it is clear that 
this kind of experiment is very costly in terms 
of both time and money. 

One purpose of this paper is to describe an 
abbreviated approach to estimating Q and K 
based on Equation 3, which requires the 
determination of only two values of d B(50 ). 
The logic of the method is as follows. If the 
smaller of the two reinforcers is delivered 
immediately (i.e. d A ~ 0), Equation 3 becomes 


d B ( 50) — 


1 
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Q./qA-Q./q B 
1 + Q/q B _' 


(3a) 


Equation 3a is the limit of Equation 3 as d A —» 
0. If two indifference delays are determined 
using different pairs of reinforcer sizes (q A i, 
q B i, and q A 2 , qiti) > then the ratio of the 
indifference delays is 


d B ( 50)i _ l/?ai — 1 /<?bi _ 1 + Q_/qm 

d B ( 5 oyi i/qA2 — i/qm 1 + Q/q B 1 


If reinforcer sizes are selected such that (1 / q A 1 
- l/?iii) = (1/?A2 “ 1 /qm). Equation 4 
simplifies to 


d B ( 50)1 _ 
d B ( 50)2 

< 2 = 


1 + Q! qm 

1 + ( 2 / q B 1 ’ 

diJ(50)l /dfl(50)2 — 1 

1/?B2 — (d B ( 50)1 / ds^f) / q B \ 


(4a) 


Q_ may thus be estimated empirically from the 
ratio of the indifference delays, and this 
estimate of f) may be substituted into Equation 
3a in order to derive an estimate of K for each 
value of c^( 50 ). 

The derivation of Equations 3 and 4 is based 
on an assumed hyperbolic relationship be- 
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tween value and reinforcer size, as postulated 
in Equations 2a and 2b (Ho et al., 1999). 
However, a simpler version of Equation 4 may 
be derived from Equation 1, which assumes 
strict proportionality between value and rein¬ 
forcer size. In this case, the ratio of two 
indifference delays is given by 

<4(50)1 _ <?B1 — <?A1 <7A2 ^ 

<4(50)2 <7A1 <?B2 “ <?A2 

Equation 5 contains no free parameters, and 
therefore yields a specific numerical predic¬ 
tion for the ratio of the two indifference 
delays. A second aim of this experiment was to 
examine whether the empirically obtained 
ratios of the indifference delays would be 
compatible with this prediction. 

In the present experiment, indifference 
delays were determined using an adjusting- 
delay schedule (Mazur, 1987). In this schedule 
the delay to the larger of two reinforcers varies 
in accordance with the subject’s choice. For 
example, if, in a block of trials, the subject 
shows a preference for the larger reinforcer 
(B), the delay to that reinforcer is increased in 
the following block; conversely, if it shows a 
preference for the smaller reinforcer (A), the 
delay to Reinforcer B is reduced in the 
following block. The principal dependent 
variable, the adjusting delay to the larger 
reinforcer (d B ), is seen to oscillate during an 
extended period of training, the amplitude of 
oscillation gradually declining as d B approach¬ 
es a quasi-stable value; this quasi-stable value of 
d B is usually taken to represent the indiffer¬ 
ence delay, <4(50) (Mazur 1987, 1988). Adjust¬ 
ing-delay schedules have been used extensively 
in behavior analytic studies of intertemporal 
choice (e.g. Green, Myerson, Shah, Estle, & 
Holt, 2007; Mazur, 1987, 1988, 1994, 1995, 
1996, 2000, 2005), but less often in neurobe- 
havioral experiments (da Costa Araujo et al., 
2009; Mobini et al., 2000). A potential advan¬ 
tage of adjusting-delay schedules in neurobe- 
havioral investigations of intertemporal choice 
is that, in addition to generating quasi-stable 
indifference delays, the pattern of oscillation 
of the adjusting delay may also provide 
information about the effects of neurobiolog- 
ical interventions on the organism’s adapta¬ 
tion to changing delays to reinforcement. An 
additional purpose of this paper is to describe 
a novel way of quantifying transitional behav¬ 


ior in the adjusting-delay schedule based on 
analysis of the power spectrum of cyclical 
changes in the adjusting delay. 

METHOD 

Subjects 

Twelve experimentally naive female Wistar 
rats (Charles River UK) approximately 

4 months old and weighing 250-300 g at the 
start of the experiment were used. They were 
housed individually under a constant cycle of 
12 hr light and 12 hr darkness (light on 0600- 
1800 hr), and were maintained at 80% of their 
initial free-feeding body weights throughout 
the experiment by providing a limited amount 
of standard rodent diet after each experimen¬ 
tal session. Tap water was freely available in the 
home cages. 

Apparatus 

The rats were trained in standard operant 
conditioning chambers (CeNeS Ltd, Cam¬ 
bridge, UK) of internal dimensions 25 X 25 X 
22 cm. One wall of the chamber contained a 
central recess covered by a hinged clear Perspex 
flap, into which a peristaltic pump could deliver 
a 0.6 M sucrose solution. Two apertures situated 

5 cm above and 2.5 cm to either side of the 
recess allowed insertion of motorized retract¬ 
able levers (CeNeS Ltd, Cambridge, UK) into 
the chamber. The levers could be depressed by 
a force of approximately 0.2 N. The chamber 
was enclosed in a sound-attenuating chest with 
additional masking noise generated by a rotary 
fan. No houselight was present during the 
sessions. An Acorn microcomputer pro¬ 
grammed in Arachnid BASIC (CeNeS Ltd, 
Cambridge, UK) located in an adjoining room 
controlled the schedules and recorded the 
behavioral data. 

Procedure 

The experiment was carried out in accor¬ 
dance with UK regulations governing experi¬ 
ments on living animals. 

At the start of the experiment the food 
deprivation regimen was introduced and the 
rats were gradually reduced to 80% of their 
free-feeding body weights. They were then 
trained to press two levers (A and B) for the 
sucrose reinforcer (50 pi, 0.6 M), and were 
exposed to a discrete-trials continuous rein- 
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forcement schedule in which the two levers 
were presented in random sequence for three 
sessions. Then they underwent daily 42-min 
training sessions under the discrete-trials 
adjusting-delay schedule for the remainder of 
the experiment. Each experimental session 
consisted of seven blocks of four trials. The 
trials were 90 s in duration. The first two trials 
of each block were forced-choice trials in 
which each lever was presented alone in 
random sequence. The other two trials were 
free-choice trials in which both levers were 
presented. The beginning of each trial was 
signaled by illumination of the central light 
above the reinforcer recess. After 2.5 s the 
lever or levers (depending on the type of trial) 
were inserted into the chamber. When a lever 
press occurred, the lever(s) were withdrawn, 
the central light was extinguished, and the 
light located above the lever that had been 
depressed was illuminated. This light re¬ 
mained illuminated until the delivery of the 
reinforcer, and was then extinguished. The 
chamber remained in darkness until the start 
of the following trial. If no lever press occurred 
within 5 s of the lever (s) being inserted, the 
lever(s) were retracted and the central light 
extinguished. (This seldom happened except 
during the first few training sessions.) A 
response on Lever A resulted in immediate 
delivery of the smaller reinforcer, of size q A 
(i.e. d A ~ 0). A response on Lever B initiated a 
delay d B whose duration was increased or 
decreased systematically from one trial block 
to the next as a function of the choices in the 
prior block; at the end of this delay the larger 
reinforcer, of size q B , was delivered. The 
positions of Levers A and B (left vs. right) 
were counterbalanced across subjects. 

In each block of trials, the delay to the larger 
reinforcer, d B , was determined by the rat’s 
choices in the free-choice trials in the preced¬ 
ing block. If Lever A was chosen in both free- 
choice trials of block n, d B was reduced by 20 % 
in block n+ 1; if Lever B was chosen in both 
free-choice trials of block n, d B was increased 
by 20% in block n+ 1; if Lever A and Lever B 
were each chosen in one free-choice trial in 
block n, d B remained unchanged in block n+ 1. 
The value of d B in the first block of each 
session was determined in the same way by the 
choices made in the final block of the previous 
session. Maximum and minimum values of d B 
were set at 60 s and 0.75 s. 


The experiment consisted of two Phases (I 
and II), the first lasting 100 sessions and the 
second 40 sessions. There were two experi¬ 
mental Conditions (1 and 2); for half the rats 
Condition 1 was in effect in Phase I and 
Condition 2 in Phase II; for the other rats the 
order of conditions was reversed. In Condition 
1 , the sizes of the two reinforcers (volume of 
0.6 M sucrose solution) were q A \ = 25 pi and 
qm = 100 pi; in Condition 2, the sizes of the 
reinforcers were q A 2 = 14 pi and q^ = 25 pi. In 
the first block of the first session of each phase, 
d B was set at 0.75 s. 

Experimental sessions were carried out 
7 days a week, at the same time each day, 
during the light phase of the daily cycle 
(between 0800 and 1400 hr). 

Data Analysis 

Indifference delays and parameter estima tion. For 
each rat, the mean value of d B in the last 10 
sessions of each phase was taken as the 
indifference delay, d n(A(r) . These data were 
analyzed by a two-factor analysis of variance 
[condition (1,2) X order of condition (1-first 
vs. 2 -first)] with repeated measures on the 
former factor. As this analysis showed no 
significant main effect of order and no 
significant order X condition interaction, the 
order factor was ignored in all further treat¬ 
ment of the data. The ratio of the values of 
d B ( 50 ) obtained under the two conditions was 
calculated for each rat, and these values were 
used to calculate estimates of (3 as described in 
the Introduction, using the formula 

, _ d B ( 50) 1 / d B ( 50)2 — 1 

1/25 —(d B( 50)i/dB(50)2)/100 

100 and 25 being the sizes of Reinforcer B 
(volumes of 0.6 M sucrose, in pi) in the two 
conditions of the experiment (q m and <p a , 
respectively). This estimate of Q_ was used to 
derive an estimate of K for each rat by 
substitution into Equation 3a. 

The obtained ratios of the indifference 
delays were also compared with the ratio 
predicted on the basis of an assumed linear 
relation between reinforcer size and instanta¬ 
neous value. Based on Equation 1, the 
indifference delay is 
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Substituting the actual reinforcer sizes used in 
this experiment into this equation, the ratio of 
the indifference delays should be 

<*/j( 50 )i _ 100 — 25 14 

<to(50)a “ 25 25-14 “' ' 

(cf. Equation 5: Introduction). The obtained 
ratios were compared with this theoretical 
value using a /-test. 

Transitional behavior. In order to characterize 
the pattern of oscillation of d B during the 
course of training, a power spectrum analysis 
was carried out on the values of d B obtained in 
each trial block during each phase of the 
experiment. The method is illustrated in 
Figure 1. Plots were obtained of log d B versus 
blocks of trials (Phase I, 700 blocks; Phase II, 
280 blocks: see upper panel of Figure 1). 
These data, expressed as deviations from the 
mean value of d B , were subjected to a Fourier 
transform (Spike-2, version 4.23: Cambridge 
Electronic Design, Ltd) in order to derive 
power spectra (power vs. frequency: see lower 
panels of Figure 1). The reciprocal of the 
frequency is the cycle time (period) of 
oscillation of d B , in blocks. The power of the 
dominant frequency of the spectrum within 
the frequency range of 0.01 (period = 100 
blocks) and 0.04 (period = 25 blocks) and the 
length of the period corresponding to the 
dominant frequency were derived for each rat 
in each phase of the experiment (da Costa 
Araujo et al., 2009). Inspection of the data 
indicated that the amplitude of oscillation of 
d B declined during the 100 sessions of Phase I. 
This impression was tested by comparing the 
power spectra derived from the first and final 
280 blocks of trials of Phase I; comparisons 
were also made between the spectra derived 
from the final 280 blocks of Phase I and the 
280 blocks that comprised Phase II. 

RESULTS 

Indifference Delays and Parameter Estimation 

Figure 2 (left-hand panel) shows the values 
°f 50 ) derived from the last 10 sessions of 
each phase. In all but 1 of the 12 rats, the value 
of <5*8(50)l (Condition 1: q Ai = 25 pi, q m = 100 pi) 
was higher than that of d B ^0)2 (Condition 2: qA 2 
= 14 pi, q B 2 = 25 pi). Analysis of variance 
indicated that there was a significant effect of 
condition (/jjo = 28.3, p < .001), but no 


significant effect of the order of conditions 
(F\ ,io = 1.3, p > .1) and no significant 
interaction (F 110 = 2.7, p > .1). Accordingly, 
the data from all 12 rats were pooled in all 
subsequent analyses. Figure 2 (right-hand pan¬ 
el) shows the ratios of the two values of d B (sop 
the horizontal line indicates the ratio predicted 
on the basis of an assumed linear relation 
between reinforcer size and reinforcer value 
(3.81: see Data Analysis section in the Method). 
The observed ratio (mean ± SEM: 2.34 ± 0.19) 
was significantly lower than the predicted ratio 
(tn = 7.6, p < .001). 

The ratios of the d B ( 50)S were used to 
compute estimates of the two parameters of 
Equation 2, (land K. The results of this analysis 
are shown in Figure 3. There was one clear 
outlier in the case of both parameters, this 
being the rat that showed no difference 
between the d B ^ 50 )S in the two conditions (see 
above). The parameter that expresses sensitivity 
to reinforcer size (0 was derived by substitu¬ 
tion of the ratio of the indifference delays into 
Equation 4a. The group mean value of Q_ (± 
SEM) was 113.8 ± 27.9 pi. Estimates of the delay 
discounting parameter ( K) were derived by 
substituting each rat’s estimated value of (4 in to 
Equation 3a. The group mean value (± SEM) 
was 0.082 ± 0.012 s~\ 

Transitional Behavior 

In all 12 rats, the adjusting delay to the larger 
reinforcer, d B , showed an oscillating pattern of 
change during the early stages of training, the 
amplitude of the oscillations tending to decline 
during extended training (see Figure 1 for an 
example; the data from all 12 rats are shown in 
the Appendix). Power spectra were derived for 
each rat’s data from three segments of training: 
the first and last 280 trial blocks of Phase I and 
the 280 blocks of Phase II. The power in the 
dominant frequency band and the period 
corresponding to the dominant frequency from 
each segment are shown in Figure 4 (upper 
panels: rats exposed to Condition 1 \q M = 25 pi, 
q B \ = 100 pi] in Phase I and Condition 2 [q A t = 
14 pi, q B g = 25 pi] in Phase II; lower panels: rats 
exposed to the two conditions in the reverse 
order). 

In both groups of rats, power in the 
dominant frequency band (Figure 4, left pan¬ 
els) was greatest in the initial segment of Phase 
I. Analysis of variance showed a significant 
main effect of segment (F 2 2 o = 29.7, p < .001), 
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Fig. 1. Example of one rat’s performance on the adjusting-delay schedule during the entire experiment, illustrating 
the methods of data analysis. Upper graph: Adjusting delay to the larger reinforcer (d B s) plotted against blocks of trials in 
the two phases of the experiment. In Phase I (trial blocks 1-700), the sizes of the reinforcers (q A , q B ) were 25 and 100 pi of 
a 0.6 M sucrose solution; in Phase II they were 14 and 25 pi. The segments A, B, and C, demarcated by the broken lines, 
comprise the first (A) and final (B) 280 blocks of Phase I and the 280 blocks of Phase II (C) that were used in the Fourier 
transform analysis. The short horizontal lines indicate the mean values of d B in the final 10 sessions (70 blocks) of the two 
phases ( Ai(. : ,o) i and d n( r ttl) ^). Lower panels: Power spectra derived from Fourier transform analysis of the d B data from 
segments A, B, and C (see above). Power is plotted against frequency (blocks^ 1 ). The period of oscillation corresponding 
to the dominant frequency band, and the power within that band, are shown for each segment. 


but no significant effect of group (ib,io = 2.9, 
p > .05) and no significant interaction {F 22 o = 
4.1, p > .05). In both groups, power was 
significantly lower in the last segment of Phase 
I and in Phase II compared to the initial 


segment of Phase I. The period corresponding 
to the dominant frequency (Figure 4, right 
panels) showed no significant effects of seg¬ 
ment or group, and no significant interaction 
(all Fs < 1). 
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Fig. 2. A. Values of d B (& 0) (s) obtained under the two conditions. Columns are group mean data; connected points 
are data from individual rats. d B ( 50 ) was significantly longer under Condition 1 (reinforcer sizes: q A = 25 pi, q B = 100 pi) 
than under Condition 2 (reinforcer sizes: q A = 14 pi, q B = 25 pi). B. Ratio of the values of rf B / 50 ) obtained under the two 
conditions. Column shows the group mean ratio (+SEM); open circles show data from individual rats. Horizontal broken 
line shows the expected ratio based on the assumption that reinforcer value is linearly related to reinforcer size (see text). 




Fig. 3. Estimates of the parameters of Equation 2 expressing the rate of delay discounting, K (s -1 ), and sensitivity to 
reinforcer size, Q (pi of the 0.6 M sucrose solution). Columns are group mean data (+SEM); open circles show estimates 
for individual rats. (In the case of both parameters, the outlier is the rat that produced the lowest ratio of the two values 
of d B(so) : see Fig. 2.) 
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POWER IN DOMINANT 
FREQUENCY BAND 


100 -i 





PERIOD CORRESPONDING TO 
DOMINANT FREQUENCY 




Fig. 4. Results of the power spectrum analysis in three segments of the experiment (see Fig. 1). Left panels: power in 
the dominant frequency band; right panels: period of oscillation corresponding to the dominant frequency. Upper panels: 
data from the rats that were exposed to Condition 1 (open columns) first and Condition 2 (shaded columns) second; 
lower panels: data from the rats that were exposed to the conditions in the reverse order. In Condition 1, the sizes of the 
reinforcers ( q A , q B ) were 25 and 100 |il of a 0.6 M sucrose solution; in Condition 2, they were 14 and 25 pi. Columns show 
group mean data (+ SEM). In both groups, power was significantly less in the final segment of Phase I and in Phase II 


than in the initial segment of Phase I (* p < .05). The 
three segments. 

Figure 5 shows the individual-subject data 
and the mean data from the rats in both 
groups in the terminal segment of Phase I and 
the segment comprising Phase II. Compari¬ 
sons between the two conditions showed that 
power was significantly higher in Condition 2 
than in Condition 1 (hi = 3.7, p < .01), but 
there was no significant difference between 
period in the two conditions (t < 1). 

DISCUSSION 

Indifference Delays and Parameter Estimation 

The quasi-stable adjusting delays seen dur¬ 
ing the last 10 days of training under each 
condition were taken as indifference delays, 


period of oscillation did not vary significantly across the 

d B ( so)- The value of d B(50 ) was higher in 
Condition 1, when the reinforcer sizes were 
25 and 100 pi of the sucrose solution, than in 
Condition 2, when they were 14 and 25 pi. This 
was an expected result, because d B ( 50 ) is 
assumed to depend on the relative instanta¬ 
neous value of Reinforcer B, which was higher 
under Condition 1 than under Condition 2 
(cf. Equation 3). 

The ratio of the indifference delays, 4/;(-,o) i/ 
d B { 50 ) 2 > was compared with a predicted value of 
3.81, which was calculated from the physical 
sizes of the reinforcers, based on the assump¬ 
tion that instantaneous reinforcer value is 
linearly related to reinforcer size (cf. Equation 
1). The observed ratios were consistently 
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POWER IN DOMINANT 
FREQUENCY BAND 



PERIOD CORRESPONDING TO 
DOMINANT FREQUENCY 



Fig. 5. Results of the power spectrum analysis. Comparison of power in the dominant frequency band and period of 
oscillation corresponding to the dominant frequency seen under Conditions 1 (open columns; reinforcer sizes: q A = 
25 Lll. q B = 100 fj!) and 2 (shaded columns; reinforcer sizes: q A = 14 pi, q B = 25 pi). Columns show group mean data; open 
circles show data from individual rats. Power was significantly higher under Condition 2 than under Condition 1 (* p < 
.05); there was no difference between the period of oscillation under the two conditions. 


smaller than 3.81, suggesting a nonlinear 
relation between size and value. A nonlinear 
relation is assumed by the model of inter¬ 
temporal choice proposed by Ho et al. (1999), 
which formed the basis of the method adopted 
here to derive numerical estimates of discount¬ 
ing parameters. According to this model, 
instantaneous reinforcer value is hyperbolical- 
ly related to reinforcer size, the relation being 
defined by a single free parameter, (l which 
specifies the reinforcer size corresponding to 
the half-maximal value. Note that other 
nonlinear size/value functions have recently 
been proposed, for example, by Killeen (2009) 
and Pine et al. (2009). 

The present experiment employed two pairs 
of reinforcer sizes that allowed () to be 
determined from the ratio of the two indiffer¬ 
ence delays (Equation 4a). It is of interest to 
compare the value of Q obtained using this 
tactic (mean = 113.8 pi) with a value of Q 
derived using a different approach. Rickard et 
al. (2009) trained rats under a progressive- 
ratio schedule using a wide range of reinforcer 
sizes (different volumes of a 0.6 M sucrose 


solution). Response rates in successive ratios 
were analyzed using an equation derived from 
Killeen’s (1994) “mathematical principles of 
reinforcement”. The “specific activation” 
parameter of Killeen’s model (a), which is 
presumed to reflect the incentive value of the 
reinforcer, was a monotonically increasing 
function of reinforcer volume. A hyperbolic 
function was fitted to the relation between a 
and reinforcer volume, from which it was 
determined that the value of Q was 158.9 pi. 
The fact that these radically different methods 
yielded numerical estimates of Q that were in 
same order of magnitude inspires some 
confidence in the general utility of this 
parameter as a descriptor of sensitivity to 
reinforcer size. 

By substituting the estimates of Q into 
Equation 3, it was possible to derive estimates 
of the delay-discounting parameter, K The 
mean value of K (0.082 s J ) was similar to 
values of this parameter obtained in previous 
experiments with rats (approximately 0.1 s -1 : 
Green, Myerson, Holt, Slevin, & Estle, 2004; 
Mazur, 2007; Mazur & Biondi, 2009). 
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Methodological Considerations 

The derivation of Equation 4 entails the 
simplifying assumption that when no delay is 
scheduled for Reinforcer A, d A = 0. In fact, a 
brief delay necessarily occurs between the 
initiation of the reinforcer delivery and the 
subject’s consumption of the reinforcer. In¬ 
formal observation of rats trained under the 
present procedure suggests that this delay is in 
the order of half a second, and we therefore 
think it unlikely that the approximation d A = 0 
significantly compromises the validity of Equa¬ 
tion 4. Moreover, there is no measurable delay 
between the response on Lever A and the 
presentation of exteroceptive stimuli associat¬ 
ed with reinforcer delivery, which presumably 
acquire some conditioned reinforcing proper¬ 
ties. 

It must be pointed out that the algebraic 
substrate of the present method imposes some 
restrictions on the range of reinforcer sizes 
that may be employed. For example, in the 
present experiment we set qm = q A \ (25 pi) 
and q B \ = 4 ,q M (100 pi); the value of q A2 
required in order to preserve the equality ( 1 / 
?ai - l/?iii) = (1/^42 “ l/fe) was approx¬ 
imately 14 pi. Other reinforcer sizes might 
have been used, but the choice is not limitless. 
For example, instead of determining the 
required value of q A2 for given values of q B \, 
q A i and q B 2 , the value of q A 2 might have been 
preselected, and the required value of qm 
calculated accordingly. In this case, when q m 
= q A 1 , the chosen value of q A2 would have had 
to be less than q B 2 / 2 , otherwise an appropriate 
value of q B i could not have been found (e.g., if 
qA 2 = 25, and q m = q A 1 = 50, q m = °°). 
Choosing convenient values of q A 1 , q B \ and q B2 
and calculating the required value of q A2 
circumvents this limitation. Figure 6 A shows 
the required values of q A2 for a range of values 
of q B 2 , when q m is set at 100 and q A \ is set at 
12.5, 25 or 50; the points indicate the values 
corresponding to the particular condition 
used in the present experiment, in which q A \ 
= qm- The figure shows that despite the 
limitation outlined above, the method can in 
principle accommodate a broad range of 
reinforcer sizes. 

The equations also impose constraints on 
the range of d B ( 50 ) ratios that can generate 
meaningful values of the parameter Q. Inspec¬ 
tion of Equation 4a shows that the range of 
allowable ratios has a lower boundary of 1.0 


and an upper boundary defined by q m and q lf2 - 
If (d B (50)i/d B ( 50 ) 2 )/ qm = 1/qm, the recovered 
value of Q_ is °°; higher d B ^ 0 ) ratios yield 
negative values of Q, which are, of course, 
meaningless. The relation between Q and 
d B ( 50 )i/d B ($ 0)2 is illustrated in Figure 6 B. Set¬ 
ting q m at 100 and qm = 25 (the values used in 
the present experiment), meaningful values of 
Q require d B ( 50 ) ratios < 4 (for q K2 = 50 the 
upper limit is < 2; for qm = 12.5, it is < 8 ). 
Within a substantial proportion of the range of 
allowable ratios (approximately 1.5-3.5 in the 
present instance), the relation between log Q_ 
and the ds( 50 ) ratio is approximately linear. As 
shown in Figure 6 B, the data from 11 of the 12 
rats in this experiment fell within this band. 
The nature of the relation between the d B ( 50 ) 
ratio and () dictates that small changes in the 
size of the ratio will tend to produce larger 
changes in Q at higher ratio sizes than at lower 
ratio sizes. This suggests that it may be 
appropriate for statistical tests on values of Q 
derived using the present method (for exam¬ 
ple in experiments examining the effect of 
neurobiological interventions on this parame¬ 
ter) to be carried out on logarithmically 
transformed parameter values. 

It must be emphasized that the constraint on 
the range of allowable d mA()) ratios is theoretical 
rather than methodological. In other words, it 
is not peculiar to the present application of 
Equation 4; rather, it arises from the assump¬ 
tions underlying Equations 2a and 2b, which 
form the basis of Ho et al.’s (1999) multiplica¬ 
tive hyperbolic model. A more stringent con¬ 
straint is imposed by Equation 1, which does 
not incorporate a size-sensitivity parameter. 
When applied to the present protocol, Equa¬ 
tion 1 specifies a d B ( 50 ) ratio of exactly 3.81. As 
discussed above, the present data do not 
comply with this specification. They are, how¬ 
ever, compatible with the limits imposed by 
Equation 4, and are therefore compatible with 
the multiplicative hyperbolic model. It is 
possible, of course, that future applications of 
the present method may reveal empirical d B ( 50 ) 
ratios that are incompatible with Equation 4; 
such an occurrence would constitute a refuta¬ 
tion of the underlying model. 

Transitional Behavior on the Adjusting- 
Delay Schedule 

In agreement with previous reports (e.g. da 
Costa Araujo et al., 2009; Mazur, 1987), the 
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^B(50)d^B(50)2 


Fig. 6. A. Relation between the required size of the smaller reinforcer in Condition 2 of the present method (<£ 42 ) f° r 
a range of sizes of the larger reinforcer (q B 2 ). The three curves show the relation between q ^2 and qm f° r three chosen 
sizes of the smaller reinforcer in Condition 1 {q A \ — 12.5, 25 and 50), the size of the larger reinforcer ( q B \) being set at 
100 in each case. The thick curve corresponds to the value of q A \ used in this experiment (25 |il of a 0.6 M sucrose 
solution). The points indicate the required values of q A ^ for the particular case of q A \ = q Bi , adopted in the present 
experiment; when q A \ = q^ — 25, q A2 ^ 14. B. Relation between the recovered value of the size-sensitivity parameter Q 
and the ratio of the two indifference delays (^( 50 ) 1 /^( 50 ) 2 ) i n the present method. The curve shows the relation between 
log Qand d#( 50 )i/ d B ( 50)2 in the case of the particular reinforcer sizes used in the present experiment (q B \ = 100 and q^ = 
q A 1 = 25). Meaningful values of Q are generated by ratios between 1 and 4. It can be seen that the relation is 
approximately linear between 1.5 and 3.5. Points show the data from the 12 subjects of the present experiment; data from 
all but one of the rats fall within that range. See text for further explanation. 
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adjusting delay to Reinforcer B (d B ) showed 
cyclical changes which gradually declined in 
amplitude as training progressed. We attempt¬ 
ed to characterize the fluctuation of d B by 
using the Fourier transform to derive a power 
spectrum of the frequency of oscillation. This 
analysis showed that the dominant frequency 
of the spectrum corresponded to a period of 
oscillation of approximately 80 trial blocks. A 
similar value for the period of oscillation was 
obtained in a recent experiment investigating 
the effect of lesions of the core of the nucleus 
accumbens on behavior on the adjusting-delay 
schedule (da Costa Araujo et al., 2009). Since 
da Costa Araujo et al.’s experiment used 
qualitatively different reinforcers (food pel¬ 
lets) and different ratios of reinforcer sizes 
from those used in the present experiment, it 
seems that the period of oscillation of d B may 
be relatively insensitive to reinforcer variables. 

There was a consistent trend for the power 
of oscillation to be highest in the initial 
segment of the first phase of the experiment. 
The decline in power during Phase I presum¬ 
ably reflects some adaptation to the schedule 
contingencies during extended training (see 
below for further discussion). Interestingly, 
although there was a consistent trend for 
power in the dominant frequency band to be 
higher under Condition 2 {q A = 14 pi, q B = 
25 pi) than under Condition 1 ( q A = 25 pi, q B 
= 100 pi), there was no overall difference 
between the power of oscillation seen in the 
second phase and that seen in the final 
segment of the first phase. This suggests that 
the gradual adaptation to the adjusting-delay 
contingencies was not disrupted by the change 
in reinforcer sizes at the start of the second 
phase. 

As in our previous experiments with adjust¬ 
ing-delay schedules (da Costa Araujo et al., 
2009; Ho, Wogar, Bradshaw, & Szabadi, 1997; 
Mobini et al., 2000; Wogar, Bradshaw, & 
Szabadi, 1992, 1993), we used proportional 
rather than fixed adjustment of d B . The 
decision to use proportional adjustment was 
based on the assumption that sensitivity to 
changes in delay of reinforcement would 
conform to Weber’s law, as is the case with 
temporal discrimination in many types of 
timing schedule (Gibbon, 1977; Killeen & 
Fetterman, 1988). Weber’s Law implies that 
proportional changes should be similarly 
discriminable across a broad range of delays, 


whereas a fixed increment of, say, 1 s would be 
less discriminable if the preceding value of d B 
were 30 s than if it were 2 s. However, fixed 
changes in d B are commonly used (e.g. Green 
et al., 2007; Mazur, 1994, 1995, 1996; Pietras, 
Cherek, Lane, Tcheremissine, & Steinberg, 
2003), and it remains to be seen whether the 
use of different adjustment rules influences 
the pattern of oscillation of d B revealed by the 
power spectrum. 

Simulating Behavior on the 
Adjusting-Delay Schedule 

Adjusting-delay schedules entail complex 
contingencies, and the processes underlying 
the oscillating pattern of changes in d B remain 
conjectural at this time. We offer the following 
speculative model as a preliminary account of 
some of the processes that may be involved in 
the schedule used in this experiment. 

We start by assuming that the value of each 
outcome is determined by Equations 2a and 
2b. As there is no delay to Reinforcer A, (d A = 
0), the value of A ( V A ) depends only upon its 
size, q A , and the size-sensitivity parameter, Q. 
The value of B ( V B ), however, varies from trial 
block to trial block, due to the influence of d B , 
modulated by the delay-discounting parame¬ 
ter, K Next, we postulate that the subject 
discriminates between V A and V B , and selects 
the outcome that has the higher value at the 
moment of choice. Thus, in any trial block in 
which V B > V A , the subject selects B, resulting 
in an increment in d B in the following block; B 
will be selected repeatedly until V A > V B , at 
which point the process will be reversed. 
Figure 7A shows how d B would oscillate in 
the present protocol if behavior were following 
this simple principle. 

To make the model more realistic, we 
assume that rats’ ability to discriminate rein¬ 
forcer value is not perfect, and that standard 
psychophysical principles apply. In keeping 
with evidence from other delay-of-reinforce- 
ment paradigms (Gibbon & Fairhurst, 1994), 
we assume that rats’ discrimination of value 
depends on the ratio of two values, rather than 
the absolute difference between them. A 
simple logistic function centered on V B /V A = 
1 may be used to define the probability that B 
will be chosen: p( B) = 1/ {l+[V B /V A \' i ), where 
s defines the slope of the function. [The 
probability that A will be chosen is p( A) = 
1—j&(B).] Figure 7B shows the effect of intro- 
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SIMULATION 
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Fig. 7. Simulation of behavior under the adjusting-delay schedule, based on the model described in the text. 
Ordinates: adjusting delay to the larger reinforcer, d B (s); abscissae: blocks of trials in the two phases of the experiment (cf. 
Fig. 1). The group mean estimated values of Q and K (see Fig. 5: Q = 113.8 gl, K = 0.082 s -1 ) were used in the 
simulations. The reinforcer volumes shown in the insets correspond to those used in Conditions 1 and 2 of the 
experiment. The recovered values of d B i $ 0 > are the means of the last 70 data points of each phase. A. Simulation based on 
the assumption of perfect discrimination between the values of the two outcomes, the subject invariably selecting B when 
Y b /V a > 1, and A when V B /V A < 1. B. Simulation incorporating variability of discrimination generated by a logistic 
psychometric function in which the probability of selection of the larger reinforcer is p(B) = l/(l+[ Vs/ V^] s ); the slope 
of the function, s, was set at 2 in this simulation. C. Simulation incorporating the additional assumption that s increases 
during training, starting at 0 and approaching its asymptote s max (2, in this simulation) according to the function 
s = s max .( l — e~ n/c ), where n is the number of trial blocks and c is a learning parameter, which was set at 300 blocks 
in this simulation. 
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ducing this element of variability into choice 
between the two outcomes. 

Finally, we suppose that rats’ ability to 
discriminate reinforcer value improves with 
practice, and that the improvement takes the 
form of a progressive steepening of the 
psychophysical function that approaches an 
asymptote after extended training. Our simu¬ 
lation uses an exponential learning function: s 
= Smax-(l~e~ n/c ), where s max is the asymptote, 
n is the ordinal position of the trial block, and 
c defines the rate at which s approaches its 
asymptotic value. This function is incorporat¬ 
ed into the simulation shown in Figure 7C. 

The model captures some features of 
behavior on the schedule used in this exper¬ 
iment, although there is clearly room for 
improvement. Like the real rats, the model 
generates oscillation of d B which declines in 
amplitude during the course of training. 
Entering the empirical values of Q and K into 
the model, it was found that d B stabilized at 
values very close to the those seen in the two 
conditions of this experiment. Although tran¬ 
sitional performance varied quite widely be¬ 
tween simulations, the steady-state values of 
d B ( so) were quite consistent: Using the param¬ 
eters listed in the legend of Figure 7, 100 
iterations of the simulation yielded mean 
values of d B ($o) of 20.0 s (Condition 1: 
<7,4=25 pi; q B =\00 pi) and 8.0 s (Condition 2: 
q A = 14 pi; q B = 25 pi), the coefficients of 
variation being 0.10 and 0.12, respectively. 
These values are reassuringly close to the 
group mean values shown in Figure 4 (18.8 s 
and 8.1 s). 

The model was less successful in capturing 
the amplitude of oscillation of d B in the early 
stages of training. In other words, the simula¬ 
tion tended to underestimate the power in the 
dominant frequency band during the initial 
part of Phase I. The reason for this is 
uncertain. Two possible explanations for the 
high-amplitude oscillations seen in this exper¬ 
iment are the following. (1) The rats may have 
had a tendency to perseverate; that is, under 
conditions of uncertainty, they may have 
tended to repeat the previous response, rather 
than selecting A or B randomly, as is assumed 
by the model. This would have had the effect 
of driving d B towards the extremes of the range 
before the alternative response was selected. 
(2) Alternatively or additionally, choices may 
have been based on outcomes obtained in an 


extended sequence of trial blocks, rather than 
being determined solely by the ratio of 
reinforcer values in the immediately preceding 
block, as is assumed in the present form of the 
model. Either or both of these hypothetical 
processes could be incorporated into the 
model. However, we believe that it may be 
premature to introduce additional parameters 
on an ad hoc basis, pending experimental 
exploration of the more basic aspects of the 
model. 

Implications for Behavioral Neuroscience 

The neural mechanisms underlying inter¬ 
temporal choice have attracted considerable 
attention in recent years. A major incentive to 
research in this area has been the prospect of 
uncovering the biological bases of pathologi¬ 
cal “impulsiveness” (Carroll, Anker, Mach, 
Newman, & Perry, 2010; Sagvolden, Johansen, 
Aase, & Russell, 2005; Williams, 2010; Yi, 
Mitchell, & Bickell, 2010). It is widely believed 
that a tendency to make impulsive choices 
arises from an abnormally high rate of delay 
discounting, and therefore discovery of the 
neural underpinnings of this process may lead 
to a greater understanding of the causes of 
pathological impulsiveness and, perhaps, to 
the development of more effective treatments 
for this disabling condition. A common tactic 
in this area of research is to examine the effect 
of a neurobiological intervention on prefer¬ 
ence for the larger of two reinforcers while 
progressively increasing the delay to that 
reinforcer. A leftward displacement of the 
resulting preference function (i.e., a reduction 
of the indifference delay) is often taken as an 
index of impulsiveness, which is not infre¬ 
quently equated with an increase in the rate of 
delay discounting. Unfortunately, the logic of 
this approach is undermined if the indiffer¬ 
ence delay is determined by two parameters 
(for example, K and 0, either or both of 
which may be influenced by biological inter¬ 
ventions. Leftward displacement of a single 
preference function provides no clue as to 
whether the intervention in question has 
altered the rate of delay discounting or the 
sensitivity to reinforcer size (or both). 

One way of overcoming this difficulty is to 
obtain several indifference delays correspond¬ 
ing to a range of delays to the smaller 
reinforcer (d A ), and thereby construct the 
linear indifference function defined by Equa- 
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tion 3. The slope and intercept of this function 
may then be used to infer changes in K and Q_ 
(Ho et al., 1999; Mazur, 2006). Although this 
method has been used successfully in studies 
examining the effects of brain lesions on 
intertemporal choice (see Introduction), the 
method is very time consuming. The approach 
adopted in the present experiment, based as it 
is on only two indifference delays, considerably 
reduces the time needed to obtain estimates of 
K and <5. It may be noted that the present 
method based on Equation 4 is one of several 
ways in which K and Q might be estimated 
from two indifference delays. For example, in 
the case of choice between two reinforcers of 
sizes (j,\ and q B , indifference delays 50 )i and 
d B ( 50)2 might be obtained for two values of d A 
( d A , and d A %) ■ According to the linear indif¬ 
ference function defined by Equation 3, K may 
be determined from the formula (slope-1)/ 
intercept. With d A \ set at 0, the slope is 
defined by (d B(50)2 ~ d B(50 )i) / d A2 , allowing 
K to be calculated from the formula 
[{(fi^(50)2~ d B (50)i) / d A2 }— l]/ds(50)i, and Q 
from the formula (1-slope)/ (slope/q B — l/q A ). 

A detailed assessment of the reliability of the 
estimates of Q_ and K obtained using the 
present method remains a task for future 
research. It is likely that the estimates based 
on two data points will show greater variability 
than estimates derived by fitting linear indif¬ 
ference functions to a family of data points 
covering a broad range of indifference delays. 
However, in neurobehavioral investigations of 
intertemporal choice, specifying the exact 
numerical values of the parameters is generally 
less important than determining whether a 
particular intervention affects one or other, or 
both, of the parameters in question (e.g., 
Bezzina et al., 2007; Kheramin et al., 2002). It 
may be noted that neurobiological interven¬ 
tions could provide a means of assessing the 
reliability and validity of the present approach. 
For example, experiments based on the linear 
indifference function defined by Equation 3 
have indicated that destruction of the orbital 
prefrontal cortex results in increases in both K 
and Qj whereas destruction of the core of the 
nucleus accumbens has a selective effect on K. 
Confidence in the utility of Equation 4 would 
be greatly strengthened if the same effects of 
the lesions on the two parameters could be 
demonstrated using the present experimental 
protocol. 


The present experiment used adjusting-delay 
schedules to obtain indifference delays. This is 
not an essential requirement. For example, 
progressive delay schedules (Evenden & Ryan, 

1996) , which have been used extensively in 
neurobehavioral experiments (see Cardinal, 
Robbins, & Everitt, 2003; Winstanley, 2010), 
are equally suitable. The adjusting-amount 
schedule (Richards, Mitchell, de Wit, & Seiden, 

1997) may be particularly advantageous, as it 
has been reported to generate stable choice 
behavior within one or two sessions, as opposed 
to the many sessions needed to reach stability 
under adjusting-delay schedules (Green et al., 
2007; Richards et al., 1997). 

The utility of analyzing the power spectrum 
of oscillations of d B remains to be explored. 
The method was used in a recent experiment 
examining the effect of destruction of the 
nucleus accumbens core on performance on 
an adjusting-delay schedule (da Costa Araujo 
et al., 2009). However, this study found no 
significant effect of the lesion on the power 
spectrum. Judgement on the method may have 
to await the results of further studies examin¬ 
ing its sensitivity, or insensitivity, to a broader 
range of neurobiological interventions. One 
limitation of the approach is that a rather large 
data set is needed in order to derive a 
spectrum of reasonable frequency resolution. 
In the present experiment, 280 values were 
used in each Fourier analysis, which yielded 
spectra with a bandwidth of 0.0039 blocks -1 , 
allowing decomposition of the empirical spec¬ 
tra into 6-10 frequency bands. It remains to be 
seen whether this degree of resolution is 
adequate to detect changes in the period of 
oscillation of d B induced by neurobiological 
interventions. 
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Appendix 

Raw data from the twelve rats. 
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Fig. Al. Data from the 6 rats that were exposed to Condition 1 in Phase I and Condition 2 in Phase II (see Fig. 1 
for explanation). 
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Fig. A2. Data from the 6 rats that were exposed to Condition 2 in Phase I and Condition 1 in Phase II (see Fig. 1 
for explanation). 



















